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ABSTRACT 

Objectives: Apply 'user testing' methodology to test 
the readability of a European Public Assessment Report 
(EPAR) summary— which describes how the decision 
was made by the European Medicines Agency to 
approve a medicine. 

Design: User testing uses mixed methods 
(questionnaire and semistructured interview), applied 
iteratively, to assess document performance— can 
people find and understand key points of information. 
Setting and participants: Testing was undertaken 
with 40 members of the public in four consecutive 
rounds of 10. Inclusion criteria, matched across rounds, 
included range of ages and educational attainment. 
Tested documents: In round 1 we tested 19 key 
points of information in a printed version of the EPAR 
summary for Bondronat (a cancer medicine). This was 
then revised to address the findings, and tested in 
round 2. In round 3 we tested the summary on-screen, 
and in round 4, tested a revised on-screen version, after 
addressing findings from both rounds 1 and 3. 
Primary outcome measure: The target followed 
European guidance for medicine leaflets: for each point 
of information 90% of participants should be able to 
find, and of those, 90% able to show understanding of 
the point. 

Results: For the original EPAR summary, 6 of the 19 
points of information reached the target (both paper- 
based and on-screen). After revisions to format and 
content, using good practice in information writing and 
design, 14 and 16 points, respectively, met the target. 
The problems related to both finding (dependent on 
layout, headings and design) and understanding (words 
and sentences used, as well as design). We devised a 
new heading structure, increased use of bullet points, 
replaced difficult and technical words and divided long 
sentences. 

Conclusions: People had difficulty finding and 
understanding key messages in the summary, but user 
testing identified the problems, and application of good 
practice resulted in a revised format which performed 
well. 
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INTRODUCTION 

European Public Assessment Report (EPAR) 
summaries are designed to inform members 



ARTICLE SUMMARY 



Strengths and limitations of this study 

■ This is the first study using a performance-based 
method to assess the readability of these docu 
ments— determining whether people can find 
and understand the information they need. 

■ Only one document was tested, but it is typical 
of the nature of EPAR summaries, and most 
improvements identified were generic and could 
be applied across all such documents, 



of the general public about how the 
European Medicines Agency (EMA) assess 
the risks and benefits of a new medicine, 
before deciding to grant a licence. 1 They are 
developed with input from patient and con- 
sumer organisations, with each summary 
being reviewed for readability and public- 
friendly language by a relevant patient prior 
to publication, but no testing of their read- 
ability is undertaken with members of the 
general public. 

The summaries are based on the full 
European Public Assessment Report, pre- 
pared by the EMA for all medicines licensed 
by the Agency. This full report is said to 
"Reflect the scientific conclusion reached ... 
at the end of the centralised evaluation 
process." The full report is lengthy and 
written for professionals. Hence, the deci- 
sion to produce a summary, lay version 
"written in manner understandable to the 
public." 1 The EPAR summaries are available 
on the EMA website, but relatively little is 
known about the usefulness of these docu- 
ments for members of the public, for whom 
they are designed. 

To test their usefulness, the performance- 
based process for assessing the readability of 
documents called 'user testing' was employed. 
This method is routinely used for the manda- 
tory leaflets for patients produced by manu- 
facturers and included in every medicine 
pack. 2 3 The use of the same testing approach 
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for EPAR summaries is supported by the fact that they 
have significant similarities to patient leaflets. 4 They are 
both relatively short documents, and based on long 
complex documents designed for professionals (the 
Summary of Product Characteristics or SmPC in the case 
of patient leaflets) — but revised to meet the needs of lay 
people. User testing determines whether potential 
readers can find and understand key points of informa- 
tion, as well as generating general feedback on the extent 
to which the information meets people's needs. 5 The 
process has been successfully used and reported for a 
variety of health information for patients, including clin- 
ical trial patient information sheets, 6 medicine label 
wordings 7 and nationally produced medicines informa- 
tion booklets. 5 

This work was undertaken in the wider context of 
health literacy, which is not just the ability to read and 
understand health information, but is now acknowl- 
edged to have a wider scope. Essentially, it has three 
aspects — the ability to read and understand health infor- 
mation, the removal by healthcare systems of unneces- 
sary complexity and barriers to patient understanding 
and involvement and a wider ability to engage with the 
healthcare process. 8 In terms of EPAR summaries, the 
three aspects relate to whether people read and under- 
stand EPAR summaries, whether they contain barriers to 
patient understanding (both are the focus of this study) 
and do these documents enable people to better engage 
in decisions about medicines ( for future study) . 

The objective was to test, then revise and re test, an 
EPAR summary (in both hard copy and on-screen 
formats) using user-testing methodology to assess 
whether the target audience, members of the public, 
can find and understand the key messages. 

METHODS 

We chose to test the EPAR summary for Bondronat, which 
contains ibandronic acid, a 'bis-phosphonate' medicine. It 
is used to treat cancers when there are high levels of 
calcium in the blood, and breast cancer in particular, 
when it has spread to the bone. The Bondronat summary 
was chosen because the data were based on more than 
one trial and its size is representative of most EPAR sum- 
maries — between one and two pages long. 

User testing is a process which uses mixed methods 
to identify problems readers have with written informa- 
tion. Good practice is then applied, designed to 
address the problems identified. There are many texts 
on good practice in information writing and design, 
and two publications have pulled together such good 
practice for people writing medicines information. First, 
a UK review of medicines information for patients 9 
(commissioned by the Department of Health) included 
an information design review, informed by five experts 
in information design, which was subsequently pub- 
lished in the form of key principles to guide the devel- 
opment of consumer medicine information. 10 Second, 



the European Union (EU) guideline on readability for 
package leaflets 11 was written to ensure such informa- 
tion was accessible and understandable, and is widely 
used in the domain. These two complementary sources 
were used to guide the revisions of the EPAR summar- 
ies — each time followed by retesting to assess the effect 
of the revisions made. The process has both a quantita- 
tive component, using an administered questionnaire, 
and a qualitative component, using a semistructured 
interview, both of which generate feedback on how the 
information performs — used to revise the document, 
prior to retesting. 3 



Participants 

The method used was in line with regulatory guidance 
on user testing for package leaflets, 11 12 that is, under- 
taken with people from the target group for the docu- 
ment — the general public. They were included if they 
could speak English to native standard. They were 
recruited from the database of Luto Research, the uni- 
versity spin out company which undertook the user- 
testing interviews. The database draws on people in the 
Leeds area of the North of England, and comprises 
people who have volunteered to take part in the testing 
of health information materials. People were excluded if 
they were current or retired healthcare professionals, 
took the medicine or had the indicated condition per- 
sonally (or related conditions) or were carers of such 
people — as such people would have prior knowledge, 
and so it would not be a fair test of the information. In 
line with common practice in user testing, there were 10 
participants in each round with 

► At least three participants of each gender. 

► Participants across the age range for the medicine. 
As the medicine is used more often by middle-aged 
and elderly people were recruited, at least one 
person in each of the 40s, 50s, 60s and 70-plus age 
groups, with no more than three people under the 
age of 40. 

► No more than two higher education graduates. 

► At least two participants who either do not use 
written documents as part of their work, or who are 
currently not working or are retired. 

A new set of participants was used in each round, to 
prevent a learning effect. Participants were recruited in 
order that age, gender, education and use of literature 
(people who used written documents regularly at work 
vs those who did not, were not working or were 
retired) were equally matched across the four rounds of 
testing. 



Materials tested 

EPAR summaries are available on the EMA website in 
each of the official languages of the EU — we tested the 
English version. 
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Round 1 

The first document tested was the printed version of the 
Bondronat EPAR summary, downloaded directly from the 
EMA website on 23 August 2010 (see figure 1). 



The EPAR summary was shown to participants directly 
on a computer screen in its original format from the 
EMA website between 19 and 27 May 2011 (see 
figure 3). 



Round 3 



Round 2 



A revised hard copy version was tested, revised according 
to the short-comings identified in the first round (see 
figure 2). Both documents were printed in black and 
white on two single-sided A4 sheets and stapled in the 
top left corner. 



Round 4 



After revision, subsequent to the outcome of round 3, a 
revised version was tested, as a webpage mock up equiva- 
lent to the original website, opened in an equivalent 
Internet browser (see figure 4). 



European Mediones Agency 



Doc Ref EMA'«0?19/2009 
EMFA7H/C/I01 



I '."inli mi it 

tbandronic a* id 



KPAR summary for Ihe public 



TIUs document is a summary of the European PuNic Assessment Report (EPARl It explains htm 
the Committee for Medicinal Products for Human l.'se tCHMPi assessed the stutlies performed, to 
reach their recommendations on ho* to use the medicine 

if you need more utformation about your medical condition or your treatment, read the Package 
Leaflet {also part of the EP.-iRi or contact your doctor or pharmacist If you mom more 
tiifni nniiion on the hosts for the CHSIP recommendations, read the Scientific Discussion (also 
part of lite EPARi. 



What Is Iloudronal? 

Bondronat i* a medicine that contain* the active substance ibandronic acid It is available as a 
concentrate iliitf is made up into a volution tor infusion (dnp into a vein) and w while. oblong tablets 
(50 nig) 

W ll.lt iN MiMKlloll.lt (IV. I MM ' 

Bondronat is used ui the following w.w > 

• ;ts ;ui infusion oi .i- .i tabid 10 prevent skeletal evert < « fracture* [broken U>irv| <<i Ih-iic 
complications ret)uinu)c treatment ) in patients with breart cancer and bone mdiMaies { wlien the 
cancer lias spread lo Ihe boneX 

• .iv .in iiitiiviv.il totrc.u hypcrcakaMftiCNfcll ItVlb Ofl Ollchn hi Ihl U00d)< Mid bj tHHOOi 
Tlic medicine can only be obtained with a prescription 

Howls Bondronat used? 

Mondrotutf treatment should only be started by a doctor who has expenence in the tteatmcnt ofcancer. 
In (lie prevention of skeletal events. Bondronat i< either juven as a6-mg infusion Listing at least 15 
minutes every thtec to four uccLv or as one tablet once a day . The tabid* must always be taken alter 
the patient has not eaten amtlnng for at least six hour* overnight and :« lca?t nuiutcs bclorc tlic 
lint food or drink of ihe day They must be taken with a (uD glass of plain water (but not mineral 
water) while standing or sitting up. and they should not be chewed, sucked or crushed The patient 
tin i -t n«< lie down for one hour after taking the tablet Patients with moderate or scveie kidney 
problem* should receive Bondronat infusions at a lower dose over an hour, or the tabids every two 
days or every week. 

In the f treatment of hypercalcemia caused by tumours. Bondronat is given as an infusion of either 2 or 
I mg. depending on how severe the hypcrcalcaemia is The infusion will normally bnng the blood 
calcium level down to normal level? within a w*ek 

How dors Bondronat work? 

The active substance in Bondronat. ibandronic acid, is a bisphc*sphonatc It stops the action of tlic 
osteoclasts, the cells in the body that are involved in breaking down the bone faRM This leads to less 
bone loss The reduction of bone loss helps to make bones less likely to break, which is useful in 
preventing fractures in cancer patients with bone metastases. 



7 WMtfcrrv Crcu&. Canary V*wrf. leojen EI4 4>C. UK 
T«4 (44-201 74 18 84 00 Fat (44-20) 74 18 84 16 



£Eiroo«an M*j«cn« Agency. 2010 R*p*>*KXicn n #jPKm#i rcev»o»dtn* sour* is a*now»«oj«wi 



Figure 1 The original EPAR summary on paper (round 1). 



RaynorDK, Bryant D. BMJ Open 201 3;3:e0031 85. doi:1 0.1 1 36/bmjopen-201 3-0031 85 



3 



Open Access 



Patients with tumours can have hiuh lc\el» ol ukium in then M.mhI ick.tscd fi«»m the hone* H\ 
preventing the breakdown of hone*, liondronat ako help* to reduce the amount of calcium released 
into the blood. 

How has liondronat hven studied? 

In the prev cntion ol skeletal event* in pjticnts with breast cancer and hone metastases, liondronat has 
heen compared with placebo (a dummy treatment) in three main studies tastmp two yean: one with 
the infusions in 466 patients and two w ilh the tablet* in a total of H46 paticnU. The main measure of 
effectiveness was based <>n tlic numbei ol new hone com pi Rations Ihcsc included spine Incline*., 
fracture* outside the *pinc and any bone complication* that needed treatment w ith radiotherapy or 

Mil I'd \ 

liondronat ha* al*o been studied in (Ik Ire Jlrm.nl of h\pcicalcacmia caused h\ turnout* in iIikx Ioui- 
week studies involvmu 4 total ol patients iWdntut wjs not compared with an\ othci treatment* 
in these studies. I he main mca*urc of effevin. eness wa* the change in blood calcium level*. 

W hat benefit hu* liondronal *hown during the studies? 

liondionat wa\ moic cffcs,ti\c than placebo in p«c\citttny bone eomplications 1 1 look lon»>ci loi 
patient* on liondronat infusions or tablet* to develop a new bone complication < V) to 7d week* i ih.m 
for patient* on placebo to 48 week*) liondronat reduced the risk of having a skeletal event by 
about 40% compared with placebo 

I t«tmli%iii.it w.i» al*o etleeti\c in heating h\pei caleaenua caused b\ turnouts About a hall lo two-third* 
of the patient* responded to a J-mg dose of liondronal with blood calcium lev el* icluimng lo w itliin 
the normal range. About three-quarter* responded to a 4-mg dmc 

What is the risk associated with liondronat? 

The most common tide effect with liondronat infusion* (seen in more than I patient in 10) is pvicxia 
(lever) I 'he moil wotntuon side elicits wilh the : tablets tseen in between land 10 patient* in UK)) are 
hypocalcacmia (low blood calcium levels). d\spe|Hia (heaitbuin). nausea (levlm^ *iek). .iklomin.il 
pain (stomach ache). oe»ophagitis (inllammation of the oesophagus, the tube that leads from the 
mouth to the stomach) ami asthenia (weakness) For the full list of all side effects repotted with 
liondronat, sec the Package Leaflet. 

liondionat should not he used in people who max be h>per*en*tti\e (allergic! to ibandiomc aeid oi 
au\ ol the other ingicdicul* It must not be used in people with h\r>ocalcacmia. and should Ik. used 
with caution in patients who arc sensitive to other htsphosphonatcs. Ihc tablets must not be used in 
patients who have abnormalities of the oesophagus or who cannot aland or sit upright for at least an 
hour liondionat should not be used in children 

W h> ha* liondronat hern appro*-**!? 

I he < oiumittcc for Medicinal Products for Human ( »c (('UNA') docidod tliat liondionat s benefits arc 
greater than its risks and recommended that it be ghen marketing authorisation 

Other inform ulion about liondronat: 

The I European Commission gran ted a marketing authorisation \al id throughout the European Inion 
for liondionat on 25 June IWf» Ihe marketing author ivjImhi hoklet is to Roche Kogi«tialion I muted 
file tnarkelmv' autlioiis.iiion i> \ .did ti>i an nnlimiled |K1mhI 

Ihc full I -PAR for liondronat is avaibblc has 
I hls summur) was last updated in 01 2010. 



Figure 1 . Continued. 
Procedure 

The document was examined by two pharmacists in the 
team (DKR, DB) to identify the most important points 
of information in the document. There were 19 points 
considered to be the most critical information (see 
figure 5), under the categories: 

► Purpose of the report and its origins (2 points); 

► Nature of the medicine and its use (3 points); 

► Studies that have been undertaken (5 points); 

► Specific issues regarding the risks of Bondronat 
(4 points); 

► Decision to grant a licence (3 points); 

► Finding more information (2 points). 

A questionnaire was then designed to assess whether 
people could find and understand each key point of 
information. The questions were set in the questionnaire 



in an order that did not follow the sequence in the 
document. 

The objective of the user test in line with current 
European guidance for patient leaflets, 11 is for 90% of 
participants to find the information in the document, 
and of those, 90% to be able to show that they have 
understood it. 

Participants were interviewed individually by one of 
the three experienced trained interviewers in purpose- 
built interview rooms, and guidance for interviewers in 
the questionnaire ensured consistent conduct of the 
interviews. Consistent scoring of the responses was 
guided by 'indicative answers ' for each question; that is, 
the information from the leaflet that the participant was 
required to provide for the answer to be scored as 
understood. For example, for the question "What two 
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Summary European Public Assessment Report 
for Bondronat (also called "ibandronic acid") 



About this Summary European 
Public Assessment Report 



Who this report is for 

This "Summary European Public Assessment 
Report" contains information for the general 
public. 

• The purpose of the report is to explain 
how it was decided to make Bondronat 
available to patients in Europe. 

What this report contains 

This report describes the research which has 
been done on the medicine. It describes what 
this research tells us about: 

• the benefits of the medicine and 

• the risks of the medicine. 

It then explains how the benefits and risks 
were weighed-up before a decision was made 
to approve it. 

Who made the decision 

The decision was made by a group called the 

• "Committee for Medicinal Products for 

Human Use" or CHMP for short. 
This is a group of experts including doctors 
and pharmacists. They work for the 
government and in universities. They are not 
connected to the company that developed the 
medicine. 

Where to get more information 

This report is a shorter simpler version of the 
main report - called the "European Public 
Assessment Report" (EPAR) 

• If you want more information about how 
the decision was made, you can find the 
full report on w ww.ema.europa.eu . 

You should look particularly at the section 
called "Scientific Discussion" 

• If you want more general information 
about Bondronat. read the package leaflet 
that comes with the medicine and on 

ww w .medi ci n es .org . You can also talk to a 
doctor or pharmacist. 




1. What Bondronat is and what it is used for 

2. How Bondronat is used 

3. How Bondronat was studied 

4. Possible side effects of Bondronat 

5. Why Bondronat was approved 



1. What Bondronat is and what it 
is used for 



What type of medicine is Bondronat? 

Bondronat contains a medicine called 
"ibandronic acid". This belongs to a group of 
medicines called "bis-phosphonates" You 
need a doctor's prescription to get Bondronat. 

How is Bondronat given? 

Bondronat is given as: 

• a drip (an 'infusion') - into a vein 

• a tablet - swallowed 

What Bondronat is used for 
Bondronat is used: 

In cancers where there are high levels of 

calcium in the blood. 

• The high levels are caused by the 
tumour. 

• Tins high level is called "hyper- 
calcaemia". 

• Here Bondronat is given as a drip. 

In breast cancer to prevent fractures 
(broken bones ) or bone complications. 

• This is when the cancer has spread to 
the bone (called "metastases"). 

• I Iere Bondronat is given as a drip or a 
tablet. 

How Bondronat works 

In the body there are cells called "osteoclasts" 
These cells help break down the bone tissue. 

In cancers where there are high levels of 

calcium in the blood. 

• This calcium comes from the bones. 

• This high level of calcium can cause 
side effects. 

• By preventing the breakdow n of 
bones. Bondronat helps to reduce the 
amount of calcium released into the 
blood. 

In breast cancer patients. 

• These patients are at risk of fractures, 
when the cancer has spread to their 
bones. 

• Because Bondronat causes less bone 
loss, these fractures are less likely. 

Please turn over for more information 



Figure 2 The revised EPAR summary on paper (round 2). 



things were weighed up before the decision was made 
about whether Bondronat should be approved," the 
indicative answer was "(Bondronat's/its) benefits AND 
risks." 

The responses to the short semistructured interviews 
were audio-recorded and transcribed verbatim. We 
looked for recurring patterns of comments and selected 
key quotes which illustrated these points. 

The questionnaire and original EPAR summary were 
pilot tested with three people from a convenience 



sample to determine whether the questions worked in 
practice. 

Round 1 (original; paper) 

The questionnaire was administered to 10 people from 
the target group, interviewed individually, with each par- 
ticipant first given time to read the EPAR summary at 
their own pace. Then, using the EPAR summary sup- 
plied, they were asked to find the information relevant 
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2. How Bond ron at is 



Bondronai should only be started by a doctor 
who has experience in treating cancer. 
For high calcium levels 
Bondronat is given as a drip 

• This is either a 2 or 4 milli-gram (mg) 
drip. 

• The amount depends on how high the 
calcium levels are. 

The drip will normally bring the blood calcium 
level down to normal levels within a week. 
To prevent fractures 
To prevent fractures, Bondronat is given as: 

• 6 milli-gram (mg) drip over at least 15 
minutes - every three to four weeks 

or 

• one tablet once a day in the early morning. 

Special information about taking the tablets 

1 . The tablets have to be taken when the 
stomach is empty. This is because food 
may slop the medicine from getting into 
the body from the stomach. This means 
the best time is first thing in the morning. 

• Before taking the tablet, the patient 
must not eat anything for at least six 
hours overnight 

• After taking the tablet, the patient 
must wait at least 30 minutes before 
the first food or drink of the day. 

2. The tablets can irritate your food pipe. The 
food pipe is the tube from the mouth to the 
stomach. This means the tablets must be 
taken: 

• whole - not chewed, sucked or crushed 

• with a full glass of plain water (not 
sparkling water) 

• while standing or sitting up 

The patient must not lie down for one hour 
after taking the tablet. 

Patients w ith kidney problems 

Patients w ith moderate or severe kidney 
problems should have Bondronat infusions at a 
lower dose over an hour. If taking the tablets, 
they take them every two days or every week. 



Please see the next column 
for more information 



3. How Bondronat was 
studi 



Research studies into bone complications 

There have been 3 main research studies of 
Bondronai in preventing fractures in patients 
with breast cancer where the cancer has spread 
to the bone: 

• one study with the drip 

- in 466 patients 

• two studies with the tablets 

- in a total of 846 patients. 
What was Bondronat compared with? 

In these studies, Bondronat was compared 
with "dummy" treatment. This is where the 
drip or tablets contain no medicine (also called 
a 'placebo') These studies each lasted 2 years. 
What did the studies look for? 
The studies looked for the number of new 
bone problems or complications. These 
included spine and other fractures. Also any 
bone problems that needed radiotherapy or 
surgery. 

What did the studies show? 
The results showed Bondronat was more 
effective than dummy treatment in preventing 
bone problems. 

• Patients having Bondronat infusions or 
tablets did not get new bone problems 
until 50-76 weeks. This compared with 33- 
48 weeks for patients on the dummy 
treatment. 

• Bondronat reduced the risk of having a 
fracture or other bone problem by about 
40% compared with placebo. 

Research studies into high calcium levels 
There have been 3 main studies in the 
treatment of high calcium levels caused by 
tumours. These lasted four weeks and involved 
343 patients. 

What was Bondronat compared with? 

Bondronat was not compared with any other 

treatments in these studies. 

What did the studies look for? 

The studies looked at any changes in blood 

calcium levels. 

What did the studies show? 

The results showed that Bondronat was 

effective in reducing high calcium levels 

caused by tumours: 

• A 2 milli-gram (mg) dose made the levels 
return to normal in about half to two-thirds 
of patients 

• A 4 milli-gram (mg) dose worked in about 
three-quarters of patients. 



Figure 2. Continued. 



to each of the 19 questions and then to explain it in 
their own words (see table 1). 

For each question, the interviewer noted any com- 
ments or particular behaviour of the participants. At the 
end of the interview, participants were asked for feed- 
back about the EPAR summary. 

Round 2 (revised; paper) 

The original EPAR summary was then refined by an 
experienced health information writer (DKR), focusing 



on the difficulties identified during the testing, and par- 
ticipants' general comments about the usefulness of the 
document. These revisions were made using best prac- 
tice in information writing and design (see above). This 
revised EPAR summary was tested on 10 more people 
from the target group for this product. 

Rounds 3 and 4 (original; electronic and revised; electronic) 

As EPAR summary documents are available in an electronic 
format, two further rounds of testing were conducted 
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Possible side effects of 
Bond ron at 



Bondronat Drip 

The most common side effect with a 
Bondronat drip is: 

• High temperature or fever 

This affects more than I in 10 people. 

Bondronat Tablets 

The most common side effects with 
Bonodronat tablets are: 

• feeling weak 

• heartburn, feeling sick or stomach ache 

• low blood calcium levels 
(hypo-calcaemia) 

These affect less than 1 in 10 people. 
Another common side effect with the tablet is 
to do with the food pipe. This is the tube from 
the mouth to the stomach. 

• The food pipe may be irritated or inflamed 
and an ulcer may form. 

• This can cause severe pain in the chest 
after swallowing food or drink or feeling 
or being very sick. 

This affects less than I in 10 people. 
This is why the tablets must be taken 

• whole 

• with a full glass of water 

• the patient must not lie down in the hour 
after taking the tablet. 

It is also why the tablets must not be used in: 

• people who have problems with their 
food pipe 

• people who cannot stand or sit upright for 
at least an hour. 

For the full list of side effects of Bondronat. 
see the Package Leaflet. 

Other risks 

Bondronat should not be used in: 

• people who may be hypersensitive 
(allergic) to ibandronic acid or any of the 
other ingredients. 

• people with hypo-calcaemia 

Bondronat should be used with care in patients 
who arc sensitive to other bis-phosphonate 
medicines. 

Bondronat should not be used in children. 

Please see the next column 
for more information 



5. Why Bondronat was 



The experts on the Committee for Medicinal 
Products for Human Use (CIIMP) have looked 
at the research studies for 

• the benefits of Bondronat 

• the risks of Bondronat. 

They have weighed up the benefits and risks. 
They have decided that the benefits are greater 
than its risks. This is why they have 
recommended that it be allowed to be made 
available on a prescription. 

The European Commission granted a 
marketing authorisation valid throughout the 
European Union for Bondronat on 25 June 
1996. The marketing authorisation holder is 
Roche Registration Limited. The marketing 
authorisation is valid for an unlimited period. 

This summary was last updated in 01-2010. 



Figure 2 Continued. 



on-screen, to test both the original and then a revised EPAR 
summary. For each version, 10 people from the target 
group were interviewed, this time referring to the informa- 
tion on a computer screen rather than in printed form. 
Inclusion criteria for these rounds additionally included: 

► Five people who use computers often (at least once a 
day). 

► Five people who use computers occasionally (not 
every day) . 



We anticipated applying the learnings from the hard 
copy testing to the subsequent on-screen testing. 



RESULTS 

Forty eligible participants were interviewed in four 
rounds of 10; their characteristics in terms of gender, 
age range, educational status and use of literature at 
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Table 1 European Public Assessment Report summary paper version: original (round 1) 


and revised (round 2) 






Original (round 1) 




Revised (round 2) 






Number of points 


Target 


Number of points 


Target 


Questions 


found/understood 


met? 


found/understood 


met? 


1 Who has this report been written for? 


7/7 


No 


8/8 


No 


2 When is Bondronat used in breast cancer? 


9/4 


No 


10/7 


No 


3 What is the purpose of this report? 


7/0 


NO 


9/9 


Yes 


4 What can the tablets do to the tood pipe < 


10/0 


NO 


10/10 


Yes 


5 Who made the decision about Bondronat described in 


10/1 


No 


9/9 


Yes 


this report? 










6 What group of medicines does Bondronat belong to? 


"7 / "7 

7/7 


K 1 _ 

No 


10/10 


Yes 


7 What did Committee for Medicinal Products for Human 


10/10 


Yes 


10/10 


Yes 


Use (CHMP) decide about Bondronat? 










8 The tablets can irritate the food pipe. Which people 


9/8 


\/,_ _ 
Yes 


10/8 


No 


cannot have the tablets as a result of this? 










9 What two things were weighed up before decision made 


"7 / "7 

ill 


NO 


10/10 


Yes 


about whether Bondronat should be approved? 










10 How does Bondronat prevent fractures, or broken 


io / y 


Yes 


8/7 


No 


bones? 










1 1 In the studies looking into bone problems or 


t~ 1 A 

5/4 


No 


10/9 


Yes 


complications, what was looked at to see if Bondronat 










was working? 










12 During the studies into high calcium levels, how many 


9/9 


Yes 


10/10 


Yes 


people did the 4 mg dose work in? 










13 How many people receiving a Bondronat infusion got a 


10/3 


No 


10/8 


No 


fever? 










14 During the study of bone complications, what was 


9/9 


Yes 


10/10 


Yes 


Bondronat compared with? 










15 Suppose you want more information on how the 


y / \ 


NO 


10/9 


Yes 


decision was made about Bondronat, what should you 










UO f 










16 How many people taking the Bondronat tablet got an 


\ U / O 


NO 


10/9 


Yes 


inflamed food pipe? 










17 In the research studies of high calcium levels in the 


9/9 


\/ _ _ 

Yes 


9/9 


Yes 


blood, what was Bondronat compared against? 










18 When Bondronat was studied in the treatment of bone 


8/8 


No 


9/9 


Yes 


complications, which group of patients got a new bone 










problem first? 










19 If you are a patient and want more information about 


8/7 


No 


10/10 


Yes 


Bondronat, what should you do? 










Total number of passes 




6 




14 





work were matched across all four rounds and are the target of 90% finding and 90% of those understand- 
described in table 2. ing (see table 1). 



Round 1: original paper version 

Quantitative findings 

In the first round of testing, with the original document 



Qualitative findings 

Participants' comments on the layout of the information 



in paper format, 6 of the 19 items of information met 


were mixed; some criticised it 




Table 2 Demographics of participants in each round 




Female (F) or male (M) Age range 


Educational status* 


Use of literature at work t 


Round 1 (10) 
Round 2 (10) 
Round 3 (10) 
Round 4 (10) 


7F, 3M 29-74 
7F, 3M 22-71 
7F, 3M 26-72 
7F, 3M 21-73 


1=4, 2=4, 3=2 
1=4, 2=4, 3=2 
1=4, 2=4, 3=2 
1=4, 2=4, 3=2 


Y=3, N=7 
Y=3, N=7 
Y=3, n=7 
Y=3, N=7 


*Educational status: 1=education complete by 16 years; 2=A-level or equivalent; 3=higher educational graduate. 

tUse of literature at work: Y=yes: uses written documents regularly at work; N=no: written document not used regularly at work, not working 
or retired. 
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It's not user friendly for a start. It's more like something 
from a lecture (PI) 

I found it quite difficult to find things ...they weren't 
where I'd expect them to be under the headings (P8) 

One participant gave a more specific comment about 
the box at the top of the page 

The little box at the top with italics... people miss out 
things like that, they'll think 'Oh it's the boring bit' (P3) 

Others were more content with the layout: 

Well set out (P 10) 

Well laid out with the titles (P3) 

Most participants said they did not have a problem with 
the language used in the leaflet. Participants described 
the language as fine, okay and straight forward among 
other things. However, one participant (PI) said it was 
too technical and another said 

There's all these big words and big sentences (P5) 

Revising the EPAR summary 

The revision took account of the results from the first 
round of testing — both quantitative and qualitative, 
where it was shown that improvements were needed. 



The revision also took account of good practice in infor- 
mation writing and design (see above). 10 

There were eight questions where, of the 10 partici- 
pants, 5 or less could find and understand the informa- 
tion. Specific changes were made to address the issues 
with these points of information and are described in 
table 3. 

As well as the changes in response to specific difficul- 
ties with answering the questions, other general changes 
made in line with good practice in information writing 
and design 10 11 were 

► Information split over two columns — some readers 
struggle with long lines of text. 

► Boxes surrounding text removed — some readers 
'read round' boxed information. 

► Text and headings justified to the left ('ragged right') 
— text justified to left and right leads to unequal 
gaps between words which can hinder ease of 
reading. 

► Bold text used as it is effective in giving emphasis — 
replacing underlined, italicised or capitalised text 
(which can make reading harder) . 

► Some paragraphs organised into bullet points — helps 
to organise text for readers, and aids finding as well 
as making document look more approachable. 

► Technical or medical terms replaced with lay lan- 
guage — for example, 'hypercalcaemia' became 'high 
calcium levels'. 



rable 3 Changes made in round 1 related to the responses to specific questions 



Question 



Changes made 



2 When is Bondronat used in breast cancer? 

3 What is the purpose of this report? 

4 What can the tablets do to the oesophagus or food 
pipe? 

5 Who made the decision about Bondronat described in 
this report? 

1 1 In the studies looking into bone problems or 

complications, what was looked at to see if Bondronat 
was working? 

13 How many people receiving a Bondronat infusion got a 
fever? 

15 Suppose you want more information on how the 
decision was made about Bondronat, what should you 
do? 

16 How many people taking the Bondronat tablet got an 
inflamed food pipe? 



Original long sentence with multiple bracketing incorporated 
into 2 new subheadings, separated into dedicated bullet points 

Original text in italics in box at beginning became part of main 
document under new heading 'Who is this report for' 

In original, mentioned both in the 'How is Bondronat used' 
section and 'What is the risk' section in isolation. Changed so 
clear in both places what problem was, and how it related to 
necessary actions 

Original did not make clear the membership of CHMP — 
remedied in revised version under the heading 'Who made the 
decision?' in first section of the document 

Answer was number of new bone complications — in original in 
middle of paragraph containing variety of pieces of information 
about the study. Revision included new subheading 'What did 
the studies look for' 

Original listed side effects in long paragraph with long 
sentences. Revision bulleted the side effects, and separated 
out those relating to 'drip' and tablets 

Information moved from original boxed italicised text into a 
new subsection titled 'Where to get more information' 

Information became part of bulleted list of side effects (see 
above), and frequency simplified from 'seen in between 1 and 
10 patients in a hundred' to 'affects less than 1 in 10 people' 
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A 


Patient safety 


» lore 


pean Medicates Agency 


MM 


ledes (lata nn»v. of 


ton 

frect 
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Mi 
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■ Now H ••adrenal rased? 

Bondronat treetmenf. show* on*, be moated by a doctor who hat axp*cwtxa in 

In tha prevention of t*a*e«*i eter*s m patents n«h breast canca* and bone 
metastases. Bondronat n edher gr,en at a t mg mf*»»on lesbng at fcatl 15 
minutes every three to four week*, or at ona tablet one* a day lha tablat mutt 
always ba taken after tha patiant hat eater nothing tor at least sti hours 
overnrgnt and at leatt JO mnwn oeSor* tnc rim food or drink of tne day. with a 
full glats d pton water vvhsst needing or s*bng Tha patiant mutt not nc down 
for era hour after taking tha tabwt eetents nMh moderate or sa.era kidney 
problem t should receive Bondronat mfvsons at a torn* dose over an hour, or tha 
tablet* e»er y two days or eter? weak 

In tha treatment of tumour- induced hfperceiceemu. Bondronat is grven as an 
infuson of 2 or a mg oapendatg on vohethar the patent's hyperrakaemla « 
moderate (Below J mmoVT) or ie»ejna (abo«e J mmol/T) ThH wfl normal>r bring 
the bkwd calcium krref down to norma' lereH wRMn day* 



■ Howdoea Bo«dro«at «aa>(»7 

The actha subaunce w Bondronat Oandronic acid. <a a bitphosphonate It slops 
the action of tho oataotlattt. the ca*» r« the body that ate m.ohed in breaking 
down the bono tissue Tnas leads to less bona lass The reduction of bone toss 
heaps to make bonea KOS tkofy to Poet, which i» useful m prevents^ fractures m 
cancer petwnts wRh bone metattates. 

Patientt with tumourt can Ka»* high lereH cf cak^m m the>*' blood, released 
from the bones. By pre lessttog Bsc BtBaldonn cf bonps. Bondronat also helps to 
reduce the amount of catc«m released **to the bkod 



Figure 3 The original EPAR summary on screen (round 3). 



Importantly, the structure was revised with a focus on 
making the document easier to navigate. It was split into 
six main headings, section numbers were introduced 
(see figure 6) and an index ('In this leaflet) was added 
to aid navigation. The main headings for each of the six 
sections of the leaflet were written as white text on a 
grey band which went across the column. This clearly 
demarcates the sections, helping readers to find the 
section they are looking for. Equally, the insertion of 
subheadings aids navigation, such as 'Who is this report 
for' and 'What type of medicine is Bondronat'. 

Finally, instead of separating the information from the 
two types of studies (into how studied and what has 
been shown), the studies of bone complications and 
high calcium levels were given their own subheadings, 
and became self-contained. 



As a result, revisions were made to the original EPAR 
summary to produce the revised version tested in round 
2 (see figure 4). 

Round 2: revised paper version 

Quantitative findings 

The objective remained that 90% of participants should 
be able to find the information in the leaflet and of 
those 90% should be able to understand it. As can be 
seen from table 1, 14 of the 19 points of information 
now met this target. 

Qualitative findings 

Participants were generally positive about the layout of 
the revised EPAR summary. Comments included it was 
fine how it's laid out (PI 7) and it's logical (P20). Several 
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Figure 3 Continued. 



participants made specific reference to the headings 
and the 'In this leaflet' index as positive features of the 
document, examples of comments were 

I like how the headings are set out, which are set into sec- 
tions which are easier to find (PI 8) 

It's good the way you've got it laid out. 'In this leaflet' 
and it tells you which section's which (P13) 

It is in bullet points and its easier to read than paragraph 
after paragraph of information (P14). 

The language was well received and described as straight 
forward (PI 2) and very easy to understand (PI 8). 
Participants commented on the use of lay language, for 
example, participant 15 described the language used as 

everyday language. 



Round 3: original electronic version 

Quantitative findings 

The objective that 90% of participants should be able to 
find the information in the leaflet and of those 90% 
should be able to understand it was met for 6 of the 19 
points (see table 4). 

Qualitative findings 

As in round 1, comments on the original EPAR 
summary were mixed with regard to the layout; however, 
the layout of the electronic version seemed to be better 
received than the paper version. Positive comments 
included it looked straight forward (P23) and It was easy to 
scroll down to whichever section your question applied to (P29). 
Participant 28 was more critical and suggested more care 
and attention to what you present under these various 
headings. 
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Table 4 European Public Assessment Report summary screen versions: original (round 3) and revised (round 4) 





Original (round 3) 




Revised (round 4) 




Number of points 


Target 


Number found/ 


Target 


Questions 


found /understood 


met? 


understood 


met? 


1 Who has this report been written for? 


4/0 


K 1 _ 

NO 


10/9 


Yes 


2 When is Bondronat used in breast cancer? 


y / 1 


NO 


10/9 


Yes 


3 What is the purpose of this report? 


b / U 


NO 


9/9 


Yes 


4 What can the tablets do to the food pipe? 


HA / A 

\ u / u 


NO 


10/10 


Yes 


5 Who made the decision about Bondronat described in this 


y / u 


N 1 _ 

NO 


10/9 


Yes 


report? 










6 What group of medicines does Bondronat belong to? 


O / O 


NO 


10/10 


Yes 


7 What did the Committee for Medicinal Products for 


y / o 


Yes 


10/9 


Yes 


numan use (onivirj aeciue aooui Donaronair 










8 The tablets can irritate the food pipe. Which people 


i u / y 


Yes 


9/8 


Yes 


cannot have the tablets as a result of this? 










9 What two things were weighed up before the decision was 


o / o 


NO 


10/10 


Yes 


made about whether Bondronat should be approved? 










10 How does Bondronat prevent fractures, or broken bones? 


\ u / y 


Yes 


7/6 


No 


11 In the studies looking into bone problems or 


o / o 

O / O 


N 1 _ 

NO 


7/7 


No 


complications, what was looked at to see if Bondronat 










was working? 










12 During the studies into high calcium levels, how many 


7/7 
III 


NO 


10/9 


Yes 


people did the 4 mg dose work in? 










13 How many people receiving a Bondronat infusion got a 


-|A / A 

i u / y 


Yes 


10/9 


Yes 


fever? 










14 During the study of bone complications, what was 


1U / O 


N 1 _ 

NO 


10/10 


Yes 


Bondronat compared with? 










15 Suppose you want more information on how the decision 


7/2 


NO 


10/9 


Yes 


was maae aooui Donuronai, wnai snouiu you uo r 










id now many people xaKing me Donuronai lauiei goi an 


Q / Q 

y / o 


Vqc 

Yes 


10/10 


Yes 


inflamed food pipe? 










17 In the research studies of high calcium levels in the blood, 


7/7 
III 


NO 


9/9 


Yes 


Wlld.1 Wdo DUllUiUilcU UUIIipdltJU ctyctlllol: 










18 When Bondronat was studied in the treatment of bone 


6/5 


No 


8/5 


No 


complications, which group of patients got a new bone 










problem first? 










19 If you are a patient and want more information about 


9/9 


Yes 


10/10 


Yes 


Bondronat, what should you do? 










Total number of passes 




6 




16 



About half of the participants thought that the lan- 
guage was fine or okay. However, other participants were 
more negative. Comments included, unnecessarily formal 
and technical (P28) and it used a lot of big words (P23). 
One participant anticipated the team's revisions after 
round 1 by saying Why don 't they put the easier words down 
instead of the big words and then in brackets put the smaller 
words (F22). 

Two participants also made reference to the Package 
Leaflet, participant 28 noted that Package Leaflet, 
although mentioned, is not immediately available on the 
website. Participant 30 also commented to say that I'm rea- 
lising this is an extra leaflet, it would be nice to see the proper 
leaflet because that's probably got it a lot clearer, how to take it. 

Revising the EPAR summary 

The changes made after the first round for the hard 
copy version were largely replicated in the revised 
on-screen version, as most of the problems identified 



were similar. However, a single column format was 
retained as this worked well in an on-screen format. 

There were nine questions where of the 10 partici- 
pants, 5 or less could find and understand the informa- 
tion. Of these, five questions (2-5 and 15) were 
addressed, as a result of problems with the hard copy 
original. This left four additional points to address, 
which are described in table 5. 

Round 4: revised electronic version 

Quantitative findings 

The objective that 90% of participants should be able to 
find the information in the leaflet and of those 90% 
should be able to understand it was met for 16 of the 19 
points of information (see table 4) . 

Qualitative findings 

The layout was generally well received by participants in 
this round. It was described as nice and clear (P31) and 
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Table 5 Changes made in round 3 related to the responses to specific questions 




Questions 


Changes made 


1 


Who has this report been written for? 


Dedicated subheading 'Who is this report for" introduced 


D 


vvricu yruup ui riit^uiuiricb uucb our lurur icu Uciuny iu r 


i nib if liuriTidiiui l, luriTicny in uic bcuiiuri nuw uucb 






DUllUrUlld.1 WUlK rypUbllKJI IcU 111 lllc IlcW bUUbcUllUll VVflcU 






iypy ui iiiuuioiiiu ib dumuiumcu r 


9 


What two things were weighed up before the decision was 


Information bulleted in revised version, to aid finding and 




made about whether Bondronat should be approved? 


understanding 


18 


When Bondronat was studied in the treatment of bone 


Simplified through new subheading of 'What did the studies 




complications, which group of patients got a new bone 


show?' 




problem first? 







simple (P32). Participant 40 also mentioned the different 
sections of the leaflet and said that it was well structured 
and in different categories which all make sense. 

In general, comments received about the language 
were positive. It was described as easy to understand (P31), 
and not complicated (P39). 

There were varying views on the electronic presenta- 
tion. Participant 32 said / love being able to do it on the 
screen. Whereas another participant thought that there is 
a lot of scrolling through the screen (P40) . Other comments 
included 

It wouldn't be very much use to me reading it on a com- 
puter because I haven't got a computer (P34) 



Qualitative feedback on EPAR summaries in general 

Participants from all four rounds of testing were asked 
to provide feedback in terms of their opinions on EPAR 
summaries in general. 

The majority (32/40) of the participants interviewed 
thought that the information in the EPAR summary 
report would be useful. 

If they could fight their way through it, it would be quite 
useful (P3) 

It was good how they gave the background . . . explain the 
benefits ... so people can see a reason for taking the 
drug (P33) 



O 
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SCIENCE MEDICINES HEALTH 



An Agency of the European 



Text size: * a A Site-wide search 




Regulatory Special topics Document search News & events Partners & networks About us 
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3 Share 
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About this Summary European Public Assessment Report 
Who this report is for 

This "Summary European Public Assessment Report" contains information 
for the general public. 

• The purpose of the report is to explain how it was decided to make 
Bondronat available to patients in Europe. 

What this report contains 

This report describes the research which has been done on the medicine. 
It describes what this research tells us about: 

• the benefits of the medicine and 

• the risks of the medicine. 

It then explains how the benefits and risks were weighed-up before a 
decision was made to approve it. 

Who made the decision 

The decision was made by a group called the 

• "Committee for Medicinal Products for Human Use" or CHMP for short. 
This is a group of experts including doctors and pharmacists. They work 
for the government and in universities. They are not connected to the 
company that developed the medicine. 

Where to get more information 

This report is a shorter simpler version of the main report - called the 
"European Public Assessment Report" (EPAR) 

• If you want more information about how the decision was made, 
you can find the full report on www.ema.europa.eu . You should look 
particularly at the section called "Scientific Discussion". 

• If you want more general information about Bondronat, read the 
package leaflet that comes with the medicine and on 
www.medicines.oro . You can also talk to a doctor or pharmacist. 





AUTHORISED 


This medicine is 


approved for use in 




the European Union 



► European Medicines Agency 
concludes class review of 
bisphosphonates and atypical 
fractures 



More information on Bondronat 

► European Medicines Agency 
concludes class review of 
bisphosphonates and aty pical 
fractures (15/04/2011) 



Figure 4 The revised EPAR summary on screen (round 4). 
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H 2. How Bondronat is used 

Bondronat should only be started by a doctor who has experience in 
treating cancer. 

For high calcium levels 

Bondronat is given as a drip 

• This is either a 2 or 4 milli-gram (mg) drip. 

• The amount depends on how high the calcium levels are. 

The drip will normally bring the blood calcium level down to normal 
levels within a week. 

To prevent fractures 

To prevent fractures, Bondronat is given as: 

• 6 milli-gram (mg) drip over at least 15 minutes - every three to 
four weeks 

or 

• one tablet once a day in the early morning. 

Special information about taking the tablets 

1. The tablets have to be taken when the stomach is empty. This is 
because food may stop the medicine from getting into the body 
from the stomach. This means the best time is first thing in the 
morning. 

• Before taking the tablet, the patient must not eat anything 
for at least six hours overnight 

• After taking the tablet, the patient must wait at least 30 
minutes before the first food or drink of the day. 

2. The tablets can irritate your food pipe. The food pipe is the tube 
from the mouth to the stomach. This means the tablets must be 
taken: 

• whole - not chewed, sucked or crushed 

• with a full glass of plain water (not sparkling water) 

• while standing or sitting up. 

The patient must not lie down for one hour after taking the 
tablet. 

Patients with kidney problems 

Patients with moderate or severe kidney problems should have 
Bondronat infusions at a lower dose over an hour. If taking the 
tablets, they take them every two days or every week. 



S 3. How Bondronat was studied 



Research studies into bone complications 

There have been 3 main research studies of Bondronat in preventing 
fractures in patients with breast cancer where the cancer has spread 
to the bone: 

• one study with the drip - in 466 patients 

• two studies with the tablets - in a total of 846 patients. 

What was Bondronat compared with? 

In these studies, Bondronat was compared with "dummy" treatment. 
This is where the drip or tablets contain no medicine (also called a 
'placebo')- These studies each lasted 2 years: 

What did the studies look for? 

The studies looked for the number of new bone problems or 
complications. These included spine and other fractures. Also any 
bone problems that needed radiotherapy or surgery. 

What did the studies show? 

The results showed Bondronat was more effective than dummy 
treatment in preventing bone problems. 

• Patients having Bondronat infusions or tablets did not get new 
bone problems until 50-76 weeks. This compared with 33-48 
weeks for patients on the dummy treatment. 

• Bondronat reduced the risk of having a fracture or other bone 
problem by about 40% compared with placebo. 

Research studies into high calcium levels 

There have been 3 main studies in the treatment of high calcium levels 
caused by tumours. These lasted four weeks and involved 343 patients. 

What was Bondronat compared with? 

Bondronat was not compared with any other treatments in these studies. 
What did the studies look for? 

The studies looked at any changes in blood calcium levels. 
What did the studies show? 

The results showed that Bondronat was effective in reducing high 
calcium levels caused by tumours: 

• A 2 milli-gram (mg) dose made the levels return to normal in about 
half to two-thirds of patients 

• A 4 milli-gram (mg) dose worked in about three-quarters of patients. 



Figure 4 Continued. 



B 4. Possible side effects of Bondronat 



Bondronat Drip 

The most common side effect with a Bondronat drip is: 

• High temperature or fever 

This affects more than 1 in 10 people. 

Bondronat Tablets 

The most common side effects with Bonodronat tablets are : 

• feeling weak 

• heartburn, feeling sick or stomach ache 

• low blood calcium levels (hypo-calcaemia) 
These affect less than 1 in 10 people. 

Another common side effect with the tablet is to do with the food 
pipe. This is the tube from the mouth to the stomach. 

• The food pipe may be irritated or inflamed and an ulcer may 
form. 

• This can cause severe pain in the chest after swallowing food or 
drink or feeling or being very sick. 

This affects less than 1 in 10 people. 
This is why the tablets must be taken 

• whole 

• with a full glass of water 

• the patient must not lie down in the hour after taking the tablet. 
It is also why the tablets must not be used in: 

• people who have problems with their food pipe 

• people who cannot stand or sit upright for at least an hour. 

For the full list of side effects of Bondronat, see the Package Leaflet. 
Other risks 

Bondronat should not be used in: 

• people who may be hypersensitive (allergic) to ibandronic acid or 
any of the other ingredients. 

• people with hypo-calcaemia 

Bondronat should be used with care in patients who are sensitive to 
other bis-phosphonate medicines. 

Bondronat should not be used in children. 



H 5. Why Bondronat was approved 

The experts on the Committee for Medicinal Products for Human Use 
(CHMP) have looked at the research studies for: 

• the benefits of Bondronat 

• the risks of Bondronat. 

They have weighed up the benefits and risks. They have decided 
that the benefits are greater than its risks. This is why they have 
recommended that it be allowed to be made available on a 
prescription. 

The European Commission granted a marketing authorisation valid 
throughout the European Union for Bondronat on 25 June 1996. The 
marketing authorisation holder is Roche Registration Limited. The 
marketing authorisation is valid for an unlimited period. 

This summary was last updated in 01-2010. 





Language 


First published 


Last updated 


Q 8ondronat : EPAR - 


EN = English 






Summary for the 


09/06/2008 


08/03/2010 


public 









This EPAR was last updated on 05/05/2011 . 

► More detail is available in the Summary of Product Characteristics 



Figure 4 Continued. 

document. A few participants listed some groups of 
people who might utilise the information, for example, 
healthcare professionals (n=4) and researchers (n=2). 



I think it's good to know that somebody in authority has 
checked it out (P20) 

Those who did not think it would be useful made 
comments about the complexity of the information. 
When asked whether people who were not patients 
would find it useful, the most common response was 
that carers or people who know someone with the indi- 
cated conditions might find it useful to read the 



DISCUSSION 

The principal finding of this research is that the current 
EPAR summary format could be improved, with only 6 
of the 19 items of information found and understood 
(to the level required for package leaflets) for both the 
paper-based and screen formats. However, the revised 
format, guided by user testing with the target popula- 
tion, better met people's needs, where 14 and 16 points, 
respectively, met the target. 
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| Introduction box / Why has Bondronat been approved? (7) j 
Question 1: who the report has been written for (Purpose; P). 
Q3: the purpose of the report (P) 

Q5: who made decision about the medicine that is described in the report (Decision; D) 
Q7: what the Committeefor Medicinal Products for Human Use decided about Bondronat (D) 
Q9: what two things were weighed up before the decision was made about the approval of 
this medicine (D) 

Q15: the actionto take if wanting more information about how the decision was made about 
Bondronat (More information; Ml) 

Q19: the actionto take if you are a patient and you want more information about Bondronat 
(Ml) 

| What is Bondronat used for? (1) | 
Q2: when the medicine is used in breast cancer (Nature of medicine; N) 

| How does Bondronat work? (2) | 
Q6: the group of medicines that Bondronat belongs to (N) 
Q10: the effect that Bondronat has on bones so as to prevent fractures (N) 

| How has Bondronat been studied? (3) j 

Qll: what was looked at in studies of bone complications to see if Bondronat was working 
(Studies; S) 

Q14: what Bondronat was compared against in studies of bone complications (S) 
Q17: what Bondronat was compared against in studies of high calcium in blood (S) 
| What benefit has Bondronat shown during studies? (2) | 

Q12: how many people the 4 milligram dose worked in during the studies of high calcium in 
the blood (S) 

Q18: which group of patients got a new bone problem first in studies of bone complications (S) 
| What is the risk associated with Bondronat? (4) | 
Q4: what the tablets can do to the oesophagus or food pipe (Risks; R) 

Q8: to understand which people cannot have the tablets due to the effects of this medicine on 
the food pipe (R) 

Q13: how many people are likely to get a fever with the Bondronat infusion (R) 

Q16: how many people are likely to get an inflamed food pipe with the Bondronat tablets (R) 

Figure 5 Key points of information tested. 

As with most documents that are user tested, there were 
problems related to both finding information and under- 
standing the information. Importantly, the heading 



structure did not work well, and points of information 
need to be placed where lay people would expect to find 
them (rather than where professionals are used to placing 
them) . The addition of new numbered headings and sub- 
headings (along with the inclusion of an 'In this leaflet' 
listing near the beginning) led to improvements in per- 
formance. Important points relating to the content were 
the need to clarify the purpose of the document at the 
beginning: saying what it is, and who it is for. The key 
recommendations are described in box 1 . 

A previous study of EPAR Summaries in Denmark by 
Askehave and Zethsen 4 focused on their nature as a 
'mandatory genre'. The study involved a questionnaire 
on lay respondents' opinion of the text, alongside an 
exercise in which participants marked problem words, 
sentences or paragraphs in the document. The authors 
concluded that most participants did not think the sum- 
maries fulfil their purpose of providing information 
which is understandable and useful to lay persons. Our 
findings with a UK population concur with those in 
Denmark, and the addition of a performance-based 
testing method also pointed the way to improvements. 

LIMITATIONS 

The study was based on one EPAR summary, which was 
of a representative size for such summaries. However, an 
EPAR summary for a medicine for cancer may be sub- 
stantially different from one on a medicine for epilepsy 
or schizophrenia in terms of what the public need to 
know. This work would be enhanced by similar studies 



Original 



Revised 



Box 1 Recommendations 



What is Bondronat? 

What is Bondronat used for? 

How is Bondronat used? 

How does Bondronat work? 

How has Bondronat been studied? 

What benefit has Bondronat shown 

during the studies? 

What is the risk associated with 

Bondronat? 

Why has Bondronat been approved? 
Other information about Bondronat 



About this Summary European Public 
Assessment Report 

- Who is this report for? 

- What this report contains 

- Who made the decision 
Where to get more information 

1. What Bondronat is and what it is used for 

- What type of medicine is Bondronat? 

- How is Bondronat given? 
What Bondronat is used for 
How Bondronat works 

2. How Bondronat is used 

For high calcium levels 

- To prevent fractures 

Special information about taking the 
tablets" 

Patients with kidney problems 

3. How Bondronat was studied 

Research studies into bone 
complications 

Research studies into high calcium 
levels 

4. Possible side effects of Bondronat 

Bondronat drip 
Bondronat tablets 
Other risks 

5. Why Bondronat was approved 



: 



Figure 6 

structure. 



Original and revised heading and subheading 



General information design 

► Appropriate use of bullets, no italics or boxes 

► Two columns for paper version. 

► Clear signposting with main headings (numbered) and sub- 
headings which work for lay people (see headings structure in 
figure 2). 

► Still a need for both hard copy and on-screen versions. The 
printable version from the website needs to be designed to 
work well on paper— not just replicate what is on the screen. 

Key messages 

► Clarify the purpose of the document up front: what it is and 
who it is for. 

► Keep the information about each study (or set of studies) 
together, that is, do not separate out 'How has been studied 
and 'What benefit has been shown in studies'. 

► The general public may not be the right audience? It may be 
more usefully aimed at people who are take or are considering 
taking the medicine concerned and people from relevant 
patient groups, rather than the general public. 

Future developments 

Clarify relationship with Package Leaflet— presenting both 
together would allow removal of most of the duplicated infor- 
mation, and allow the European Public Assessment Report 
summary to focus on the studies, their findings and the 
decision-making process. 
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on different types of medicine. However, in the mean- 
time, most of the improvements identified were generic, 
and not specific to the type of medicine, and would be 
applicable to most summaries. 

We refer later in this section to studies which show that 
patients want more balanced information, including 
benefit information. However, we did not include such a 
heading in the final version of the EPAR summary, for 
example, 'What are the benefits of Bondronat?' The inclu- 
sion of such a heading may need to be studied in future 
research on these documents. 

Relationship with wider literature 

Previous studies examining health literacy have also 
found that many people have difficulty in understanding 
health information in the UK, 13 Australia 14 and the 
USA, 15 including statutory medicines information. 16 
Furthermore, a review of drug information for consumers 
and patients with a focus on the EU concluded that 'drug 
authorities' should see themselves more as a source of 
medicine information for patients. 17 This study has also 
demonstrated the concept that unnecessary complexity 
can act as a barrier to patient understanding 2 — shown 
here in the fact that a minority of the items of informa- 
tion could be found and understood. What this work 
does not show is whether the revised summaries enable a 
wider ability to become more empowered to engage with 
the healthcare process — part of the wider nature of 
health literacy 18 — this needs further study. 

What is the target group? 

Most stakeholders would agree that people should be 
able to find out how the decision to license a medicine 
was made. However, the full EPAR is never going to be a 
document which is going to be useful to lay people, and 
EPAR summaries are a laudable attempt to open up the 
medicines licensing process in the EU. Although the 
intention was that they should be understandable to the 
public, their effectiveness has not been previously tested. 
Owing to the target group, as stated by the EMA, being 
'the public', the participants in this series of tests were 
representative of the general population, weighted to 
more older people and including people of lower educa- 
tional attainment. However, one finding of the study is 
that it was not certain for some participants as to 
whether this type of information would be useful to 
them personally. Although not stated by participants, it 
could be argued that the target group should be people 
with the condition for which the medicine is used, who 
are actively involved in their healthcare, and people 
from relevant patient groups (rather than members of 
the public generally). If so, then future testing should 
be undertaken with people from these target groups. 

Relationship with package leaflet 

Some participants raised the question of the relationship 
between the EPAR summary and the Package Leaflet. It 
is clear that there is considerable overlap between the two 



documents. The EPAR summary has general information, 
also contained in the leaflet (about what the drug is, 
what it is used for and how it is used, along with some 
side effects). The rest of the document, detailing the 
studies and the decision process would not make sense 
without that background information. However, the back- 
ground information does appear to be problematic, and 
takes the focus away from the key points relating to the 
research studies and the decision process. One option 
would be to incorporate the patient leaflet (or parts of it) 
into an EPAR summary 'package'. This could comprise 
an introduction, which describes the purpose of the 
'package', and explain that the reader should first read 
the leaflet, and then the EPAR summary. The latter could 
then focus only on the studies and the decision-making 
process, that is, the information under the current head- 
ings 'How Bondronat was studied' and 'Why Bondronat 
was approved'. 

A notable difference between the EPAR summary and 
the package leaflet is that the former explicitly mentions 
the likelihood of benefit. This is generally not present in 
patient leaflets, certainly not with any numerical infor- 
mation about the benefit. However, there is increasing 
support to include more benefit information in package 
leaflets, 19 to answer the call from patients that they want 
more balanced information on which they can make 
decisions about whether a medicine is right for them. 20 
At present, including this information is a problem, 
because of the current requirement that the leaflet is 
based on the SmPC. A second problem is that the infor- 
mation needs to be presented in a way that is under- 
standable to the public. It could be argued that 
including benefit information based on the EPAR 
summary (in the revised format developed in this study) 
would solve both problems. 

This study is further evidence to support the applic- 
ability and usability of the user-testing method to 
improve a variety of health-related information. It also 
shows that paper-based EPAR summaries as well as 
on-screen versions continue to be needed by different 
groups of the population. Importantly, the information 
needs to be appropriately formatted for each medium — 
as used in this study. 

Finally, this study has shown that there is support for 
the idea of a document which describes in patient 
friendly terms, the studies on which the decision was 
made to make the medicine available in the EU. 
However, the EPAR summary document did not 
perform well in user testing, but the testing highlighted 
the problems, and application of good practice resulted 
in revised formats which performed well. 
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